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Preface 


In  March  of  1973  Professor  C.  R.  Rao  gave  a  special  series  of 
colloquium  lectures  at  Purdue  University.  The  first  three  lectures 
were  on  a  unified  theory  of  estimation  in  the  Gene  ;al  Gauss  Markov 
linear  model.  During  the  lectures,  notes  were  taken  of  the  material 
presented  by  Professor  Rao.  Inis  report  is  a  presentation  of  these 
lecture  notes  together  with  additional  details  cf  proofs  which 
Professor  Rao  kindly  supplied. 

The  responsibility  for  the  correctness  and  accuracy  of  these  notes 
lies  with  the  note  taking  committee  composed  of  Professors  Cote  and 
myself.  Dr.  T.  Santner  and  Mr.  A.  K.  Bhargava.  The  committee  thanks 
Professor  Rao  for  his  patience  in  answering  questions  which  arose  during 
the  preparation  of  these  notes.  Special  thanks  are  also  due  to  Mr.  A.  K. 
Bhargava  for  his  substantial  efforts  in  preparing  these  notes. 


£.  M.  Kliako 
Colloquium  Chairman 
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1.1  INTRODUCTION 

In  a  series  of  papers  the  lecturer  developed  two  approaches  towards 

a  unified  treatment  of  the  General  Gauss-Markoff  (GGM)  linear  model 
2 

(Y,  X(3,  0  V)  where  V,  the  dispersion  matrix  of  Y,  may  be  singular  and  X 
may  be  deficient  in  rank.  One  is  called  the  inverse  partition  (IPM)  method 
which  depends  on  the  numerical  evaluation  of  a  g-inverse  of  a  partitioned 
matrix.  Another  is  an  analogue  of  least  square  theoiy  and  is  called  unified 
least  square  (UL3)  method. 

It  may  be  noted  that  Aitken's  [1J  approach  (which  is  called  generalized 
least  squares)  is  applicable  only  when  V  is  non-singular  although  the  require 
ment  that  X  is  of  full  rank  can  be  relaxed. 

The  aim  of  these  lectures  is  to  bring  out  the  salient  features  of  tnese 
two  methods  and  to  point  out  some  interesting  features  of  linear  unbiased 
estimation  when  the  dispersion  matrix  of  the  observations  is  singular. 

1.2  STATEMENT  OF  THE  PROBLEM 
Consider  the  triplet 

(1.2.1)  (Y,  X8.  o2V) 

where  Y  is  an  nxl  vector  of  ranarvn  variables,  X  is  a  given  nxm  matrix  and  E 
is  an  unknown  mxl  vejtor.  Furthermore, 

E(Y)  =  XS 

and  D(Y)  «  o2V 
where  o2  is  unknown. 


We  refer  to  set  up  (i,2.1)  as  the  General  Gauss-Markoff  (GGM)  model.  No 
assumption  is  made  about  R(V)  and  R(X)  where  R(*)  denotes  the  rank  rf  the 
matrix  argument. 
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The  problem  is  to  estimate  8  and  o^.  An  associated  problem  is  that 
of  testing  hypotheses  e.g.  Test  H^:  P'B  =  w,  where  P  is  a  given  kxm  matrix, 
on  the  basis  of  the  given  model. 

The  classical  method  of  solving  the  above  problem  is  the  method  of 
least  squares.  Various  types  of  difficulties  can  arise  i.e.  the  parameters 
may  not  be  independent  and  the  variab’es  may  be  related  in  the  following  sense: 

(a)  R(X)  <  m 

(b)  R(V)  <  n  (>i  =0) 

If  neither  of  the  above  two  difficulties  is  present  then  a  solution  to  ihe 
problem  of  estir^tion  of  8  is  the  8  which  minimizes  (8  is  not  estimable  if  R(x)4mj 
q  =  (Y-XS) ’ V_i  (Y-X8). 

2.  PRELIMINARIES 
2.1  Notation 

The  following  notation  will  be  used  throughout. 

(a)  The  vector  space  generated  by  the  columns  of  a  matrix  X  is  represented 
b; yjtiX) . 

(b)  The  vector  spa^e  orthogonal  to^£(A)  is  denoted  by^f(Ai)  where  A1  is  a 
matrix  of  maximum  rank  with  its  columns  orthogonal  to  the  columns  of  A. 

(c)  If  V  is  a  n.n.d.  (non-negative  definite)  matrix  the  expression 

f Ip! i  =  (P’vp)i/2 

•.here  p  is  a  vector  is  called  the  V-norra  of  p. 

(c)  The  BLUE  (best  linear  unbiased  estimator)  is  the  iinear  unbiased 
estimator  with  minimum  variance. 

(e)  (X:V)  denotes  a  partitioned  matrix  and  R(X)  the  rank  of  matrix  X.  A 

matrix  with  all  zerc  entries  is  denoted  by  0. 


3 


2.2  Some  Results  on  g-inverses  of  a  Matrices 


Def.  2.2.1  Let  A  be  an  mxn  matrix.  A  g-inverse  of  A  is  an  nxm  matrix 
denoted  by  A",  satisfying  the  condition 
AA'A  =  A 

Generalize:  inverses  have  the  following  properties. 

(a)  AA'B  =  B  =  AK  i.e.^;(B)  cJX A). 

Proof: 

Sufficiency  is  obvious.  To  prove  necessity  choose 

K  =  *  ~B. 

(b)  Let  A  be  of  order  mxn  and  3e  A~  be  any  g-inverse  of  A. 


(  i)  A  general  solution  of  ti  ?  homogeneous  equation 

Ax  =  0 


x  =  (I-A~A)z, 

where  z  is  an  arbitrary  ve-  tor; 

(ii)  a  general  solution  to  a  consistent  non  homogeneous  equation 

Ax  =  y 


x  =  A~y  +  (I-A~A)z, 


where  z  is  an  arbitrary  vectov 


Proof: 


(  i)  Note  that  this  is  equivalent  to  saying  that  the  orthogonal  space 
of  A'  =^XI-A  A)  which  follows  from  the  fact  that 
A( I -A' A)  =  0 


R(I-A'A)  =  n-R(A). 
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(ii)  follows  since  a  general  solution  of  Ax  =  y  is  the  sum  of  a 

parti cular  solution  of  Ax  =  y  and  a  general  solution  of  Ax  =  0. 

(c)  The  projection  operator  on^(x)  is 

(  i)  =  X(X'X)  X',  which  is  unique  for  any  choice  of  the  g-inverse, 
when  the  inner  product  (x,y)  =  x'y,  and 
(ii)  Pj,  =  X(X'AX)  X'A,  which  is  unique  for  any  choice  of  the  g-inverse, 
then  the  inner  product  (x,y)  =  x'Ay,  A  being  a  p.d.  matrix. 

Proof  of  (i) :  By  property  (a),  we  have  X'X(X’X)  X'  -•  X'.  Then 

°.PX  =  XfX'Xrx^fX'XrX'  =  X(X'X)'X*  =  px 

so  that  P^  is  idempotent. 

Further  [(X'X)-]'  is  also  a  g-invcrse  of  X'X.  Then  by 
uniqueness  for  choice  of  g-inverse 

P^  =  X£(X'X)_] »X*  =  X(X'X)'X'  =  Px 

so  Px  is  symmetric.  Thus  ?x  is  the  projection  operator. 

Proof  of  (ii) :  The  proof  is  the  same  as  in  (1).  We  establish 
Px  is  idempotent  and  APX  is  symmetric. 

(d)  M*x)  =^[I-(X')'X’]. 

Proof:  Let  R(X’)  =  r. 


Then  X' [I- (X* ) "X1 ]  =  X' -X' (X’ ) 'X*  ^  X'-X'  =  0 


Next  we  show  that  R(I-(X')  X’]  =  n-r- 

This  follows  easily  from  the  fact  that  the  matrices  [I-(X')  X'l, 

I  and  (X')X  are  all  idempotent.  Therefore 

R[I-(X’)‘X']  =  TracefI-(X’}“X')  =  Trace  I  -  Trace  (X')'X'  =  n-r 
le)  Consider  the  equation 

(2.2.1)  AXA  =  A 

Tnen  four  alternative  representations  of  a  general  solution  to 

(2.2.1)  are,  with  P,  as  the  projection  operator  on^(A), 

A 


(  i) 

X  -  A 

+  U  -  A  AUAA 

(  ii) 

X 

II 

> 

1 

+  (I-A'A)V+W(I-AA_) 

(iii) 

X  -  *' 

+  u  -  Pa'^a 

l  iv) 

X 

If 

> 

i 

+  W(I-PA)  +  (I-PA,)  V 

whe-e  A~  is  a  particular  g-inverse  and  U,  V,  W  are  arbitrary  matrices. 
Proof: 

Verification  of  these  identities  is  straightforward  and  left  to  the 
reader. 

(f)  The  equation  AXB  =  C  has  a  solution  if  and  only  if 

(2.2.2)  AA'C  B"B  =  C. 

In  such  a  case  a  solution  is  given  by 

(2.2.3)  X  =  A'C  B'  +  Z-A'A  Z  BB_ 

where  Z  is  arbitrary. 

Proof: 

Necessity  of  (2.2.2)  follows  from  the  fact  that  if  the  equations  are 
consistent  there  exists  a  matrix  X  such  that 

AXB  =  C 

Then  AA-  C  B"B  =  AA’  AXBB'B  =  AXB  =  C.  Sufficiency  is  trivial  since 
here  A"CB"  is  clearly  a  solution.  Observe  that  X  defined  by  (2.2.3) 
satisfies  the  equation 


AXB  =  C. 
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Also  any  arbitrary  solution  X  of  this  equation  is  obtainable 
through  the  formula  (2.2.3)  by  a  suitable  choice  of  the  matrix  Z; 
for  example, 

2  =  X-A~  C  8‘ 

is  such  a  choice.  This  shows  that  (2.2.3)  provides  the  general 
solution. 

fg)  (A  generalization  of  Firher-Cochran's  Theorem.) 

k 

Theorem  2.2.1.  Let  A_  be  mxp^matrices  of  rank  r,  i  =  1,2, . . .  ,k,5  r.  =  rn. 

"  2  i  i  •  " «  i 

Then  the  following  are  equivalent: 

(  i)  A^  Aj  *  0 

(ii)  I  -  At  (aIa.)-  a: 

Proof:  Rao  and  Mitra  [4]  prove  a  more  general  result.  Theorem  2,8.1  on 
p.  33-34. 

(h)  Let  V  be  a  n.n.d.  matrix  and  X  be  any  given  matrix.  If  there  exists  a 
matrix  U  such  that 

JC(y  *  XUX')  =  ^(V:X). 

Then, 

R[X-  (V  +  Xb'X')"  X]  =  R[X'  1  • 

Proof: 

See  Lemma  5.2.2  for  the  proof. 

(i)  Def.  2.2.2.  A  matrix  denoted  by  A^^  is  said  to  be  u  minimum  N-norm 

inverse  of  A  if 

A  _ 

X  =  r«<(N)y 

is  a  solution  cf  the  consistent -equation 
Ax  -  - 

with  the  smallest  N-norm  (being  defined  as  /x'Nx)  where 

■j. 


N  is  an  n.n.d.  matrix. 


Remark  2.2.1 


(  i)  nee^  not  be  unique 

(  ii)  ^  c  iA  J 

(lll)  ^(N)  =  G 

if  and  only  if 


AGA  =  A 
(GA) *N  =  NGA 

(j)  Let  Ax  =  y  be  a  not  necessarily  consistent,  equation  then  a  matrix  denoted 
by  is  said  to  be  M-least  square  inverse  of  A  if 

*  ’  Ai(M)  * 

minimizes  the  quadratic  form 

(Ax-y)'  M(Ax-y) 

where  M  is  a  p.d.  matrix,  x  is  called  a  M-least  squares  solution 
of  Ax=y. 


Remark 

2.2.2 

(  i) 

A^^  need  not  be  unique 

(  ii) 

{A£(M)}  c  {A 

(iii) 

G  =  A~,^  if  and  only  if 

rAGA 

1  (AG) *M 

=  A 
=  MAG 

(k)  Def.  2.2.3  A  matrix  denoted  by  A*n, 

Mn 

(=A+)  is  said  to  be  a  minimum  N-norra 

M-least  squares 

inverse 

of  A  if 

x  =  A+q  is  an  M-least  square  solution  of  Ax=y  with  a 
minimum  N-norm  ,  where  M  and  N  are  p.d.  matrices. 


Remark  2.2.3 
(  i)  A+  is  unique 

(ii)  if  G  =  A+  then  following  holds  (and  conversely) 

AGA  =  A 

GAG  =  G 

(GA)fN  =  NGA  (N  p.d.) 

(AG)'M  =  MAG  (M  p.d.) 


Proof:  Let  G  =  X  , 

£(v-i) 

Then 

(2.3.1)  G'  =  [X"  ]' 

UV'1) 

From  the  definition  of  G  (Remarks  2.2.1  (iii)),  we  have 
(XG)'V'1  =  v"1  XG. 

Therefore 

XGV  =  V ( XG) * , 

and 

(G'X*)'  V  =  V(G'X'). 

Again  by  Remarks  2.2.1  (iii)  we  have 
G'  -  (X')-(y). 

Combining  (2.3.1)  and  (2.3.2)  gives  the  result. 


(2.3.2) 
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2.3.1  Application  of  Juality  Theorem 

(Another  p'ooC  of  the  Gauss-Markoff  Theorem) . 

Consider  the  following  minimization  problem.  Minimize 

(Y-X8) 1  V"1  (Y-X8). 

A  solution  to  the  above  problem  is: 

(2.3.3)  8  =  X'  .  Y 

*(V_1) 

Consequently  an  estimate  of  p'B  is  p'S  -  p'X  .  Y. 

KV  ) 

Next,  suppose  we  want  to  find  an  estimate  of  p’6  by  L'Y  such  that 

(a)  X'L  =  p  (unbiasedness) 

(b)  L'VL  =  minimum. 

A  solution  to  above  problem  is  given  by 

L  =  (X’ >M(V)  P  * 

Thus  an  estimate  of  p*8  is 
L'Y  = 

By  the  Duality  Theorem,  this  solution  can  be  written  as 
L'Y  =  p' [X-  |  ]Y. 

H\'  ) 


From  (2.3.3),  the  right  hand  side  can  be  written  as  p'3  which  is  a  least 
squares  solution. 
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2-4  Computation  of  A  (Shows  existence). 

Let  A  be  an  nxn  matrix  of  rank  r  <  n. 

(a)  If  A  is  symmetric  then  it  has  a  spectral  decomposition 

A  Wi  *  X2P2P2  +-“+  Wr 

where  Xj,  ^2'*-Ar  are  non- zero  eigenvalues  of  A  with  corresponding 
eigen-vectors  in  such  a  case 

A  =  X7P1P1  +XJP2P2  +-“+r  PrPrwith  PiPj=0  iTJ- 

(b)  If  A  is  n£l.,5ynanetric  then  it  has  a  singular  value  decomposition 
(see  [4]  p.  38,  [3]  p.  42) 

A=  VjQJ  +  X2P&  +--+  \PT% 

wnere  Pi‘P2'-*-»pr  are  the  eigen  vectors  of  AA'  and 

Ql>Q2* • • • »Qr  are  the  eigen  vectors  of  A'A 

Xi  are  the  positive  square  roots  of  the  eigenvalues  of  A’A. 
In  this  case 

A  =  ■—•  Q, P'  JL  0  pi 

A1  1  1  *r  r  r 

Remark  2.4.1  P1>P2,...,Pr  are  orthogonal  to  each  other 

Qj»Q2f''>QT  are  orthoj,  nal  to  each  other. 

i  Condition  of  Consistency 
Consider  the  CbM  model 
f3-1*1)  (Y,  Xg,cr2\&. 

It  raa>  be  noted  that  the  Gauss-Markoff  model  with  restrictions  on  the 
parameter  6 

"  (Y,XB,o2Vj;C  =  RB 


3 
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can  be  written  as  the  GGM  model 

(3.1.3)  (Ye,  Xe8,  o2Ve), 
where 

(3.1.4)  Ye-  (*):  X,  -  (*).  Ve.  (™). 

When  V  is  singular  in  (3.1.1)  there  are  some  natural  restrictions  on 
the  random  vector  Y  and  possibly  on  the  parameter  vector  8. 

One  such  restriction  on  Y  is  given  by  the  following: 

Lemma  3.1.1 

L'X  =  0,  L’V  =  C  implies  that  L'Y  =  0  with  probability  1. 

Proof:  The  conditions 

E(L'Y)  =  L'Xg  =  0 
Var(L'Y)  =  L'VL  =  0 

imply  that  L'Y  =  0  with  probability  1.  As  a  consequence  of  the 
above  lemma,  we  have: 

Theorem  3.1.1 

(a)  Y  €j?(V:X)  with  probability  1. 

This  is  called  the  consistency  of  the  model. 

(b)  R(V)  =  t  <  n,  implies  the  existence  of  an  (nxs)  matrix  K  such  that 
K'V  =  0.  Here  s  =  (n-t)  and  the  choice  of  K  =  VA  works. 

(c)  cov(K'Y)  =  a2K'VK  =  0  implies  that  K'Y  =C  (constant  vector)  with  prob.  1. 

(c)  says  there  exist  s  independent  linear  functions  of  Y  which  are  constants  with 
probability  1. 

Remark  3.1.1 

Another  way  to  state  the  above  result  is: 

Y-Yq  €>tf(V)  where  Y^  is  an  observed  value  of  Y  or  Y  =  Y^  +  VZ  where  Z  is  an  arbitrary 


vector. 
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(a;  Restrictions  on  the  random  variable  Y 
K'Y  =  C. 

Therefore  Y  lies  on  the  hyperplane  K’Y  =  C. 

We  show  that  Y  lies  on  a  hyperplane  through  the  origin. 

Let  D  =  CA.  Then 
D'K'Y  =  D'C  =  0; 

i.e.,  N'Y  =  C,  where  N'  =  D’K'.  This  implies  that 

y  tjtm. 

(b)  Restrictions  on  the  paramete _ 6 

E(K'Y)  =  K'X6  =  C. 

Therefore, 

D’K'XB  =  0(D=CX)  ~  N'XB  =  0 
where  N*  =  D’K'. 

3.2  Unbiasedness  of  a  Linear  Estimator 

Let  us  consider  the  model  (1)  and  find  the  condition  for  a  linear 
function  L'Y  to  be  unbiased  for  p'6. 

(3.2.1)  E(L’V)  =  L’XB  =  p'(B) 

.thich  must  hold  for  all  P<  such  that 

=  0 

Tnen  there  exists  a  vector  A  such  that 

L'X  -  p'  =  A'N’X 
or  p  =  X'(L-NA) 


‘Si 

% 


Tnus  we  have  the  following  lemmas. 
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Lemma  3.2.1 

A  necessary  and  sufficient  condition  that  p'B  admits  of  a  linear 
unbiased  estimator  is  that  p  t>f(A'). 

Lemma  3.2,2 

If  L'Y  is  unbiased  for  p ’ 8  then  it  is  necessary  and  suf'j*  ient  that 
there  exists  a  vector  X  such  that 

(3.2.2)  X'(L-NX)  =  p 

Lemma  3.2.3 

If  L'Y  is  an  unbiased  estimator  of  p'B  then  there  exists  a  vector  M 
such  that 

X'M  =  p 

and  L'Y  =  M'Y  with  prob.  i. 

Proof : 

Take  M  =  L-NX  (X  as  defined  by  (3.2.2)). 

Then  M'Y  -  L'Y  -  X’N'Y  =  L'Y  with  prob.  1.  Also  note  that 
X’M  =  X'L  -  X'h’X  =  p. 

r  q.e.d. 

P.emark  3.2.1 

(  i)  Note  that  when  V  is  of  full  rank  or  when  the  observation  Y  is  unknown, 
the  condition  for  unbiasedness  of  L'Y  for  p'B  is 

X'L  =  p 

which  is  usually  given  in  textbooks. 

This  is  not  true  in  general  as  (3.2.2)  shows. 

(ii)  Lemma  (3.2.3)  shows  that  the  entire  class  of  unbiased  estimators  of 

an  estimable  function  p'B  can  be  generated  by  M’Y  where  M  satisfies  the 
condition  X'M  =  p. 
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Thus  to  find  the  m.’.iimum  variance  unbiased  estimator  of  p'8  we  need  to 
determine  M  such  that 

M'VM  is  minimum 
subject  to  the  condition  X’M  =  p. 

(iii)  The  result  of  Lemma  3.2.2  is  based  on  the  knowledge  of  the  matrix  N. 
which  can  be  computed  if  V  and  a  sample  observation  on  the  r.v.  Y  are 
known. 

However  if  we  want  L'Y  to  be  unbiased  for  p'8  irrespective  of  the 
subspace  to  which  Y  may  belong  then  the  condition  is 

X’L  =  p. 

Fortunately, in  view  of  Lemma  3.2.3  the  formulae  we  develop  for  the 
BLUE  of  p'8  and  for  the  estimation  of  c'"  are  valid  no  ratter  which 
particular  subspace  Y  may  belong  to. 

4.  THE  IP.M  METHOD 
4. 1  Preliminaries 

The  Inverse  Partition  Matrix  (1PM)  Method  requires  the  computation 
of  a  g-ir.verse  of  the  partitioned  matrix 


where  V  and  X  are  defined  as  in  the  model  (3. 1.1). Once  a  g-inverse  is 

computed  by  a  suitable  procedure  we  seem  to  have  a  PanJo. a’s  box  supplying 

all  the  ingredients  needed  for  obtaining  the  BLUE'S,  theii  variances  and 

2 

covariances,  an  unbiased  estimate  of  o  ,  and  test  criteria  without  any 
further  computations  except  for  a  few  matrix  multiplications.  Thus  the 
problem  of  inferei.ee  from  a  linear  model  is  reduced  to  the  numerical 
problem  of  finding  an  inverse  (a  g-inverse)  of  the  symmetric  matrix  given 
in  (4.1.1). 


fc’e  summarize  some  results  about  g-inverse  of  a  partitioned  matrix  in 


the  following. 


Theorem  4  1.1 


Let  V,  X,  Cj,  C?,  Cy  C4,  be  as  defined  in  (4.1.1).  The  following  hold: 


(  i) 


V  X 
X'  0 


r«  c 

C1  L3 

C*  -C' 


is  another  choice  of  g-inverse. 


(  ii)  X  c3  X  =  X  =  X  C'  X 


(  iii)  V  C2  X'  =  X  q  V  =  X  C4  X'  =  x  q  X*  =  V  c*  X'  =  X  C3  v 
(  iv)  Xf  Cj  X  =  0,  V  Cj  X  =  0,  X'  Cj  V  =  0,  V  Cj  V  +  X  C3  V  =  V 
(  v)  V  Cj  V  Cj  V  =  V  C,  V  »  V  CJ  V  C,  V  *  V  C{  V 


1  ’  '1 


(  vi)  Trace  V  q  =  R(V:X)  -  R(X) 

AA 

(  vii)  (  1  is  a  g-inverse  of  (V:X) 

\V 


.....  /V  X 
(vm)  ( 

\  X*  0 

Proof: 


C  -r 


is  another  choice  of  g-inverse. 


The  result  (i)  is  proved  by  taking  transposes  of  either  side  of  (4.1.!) 
(ii)  and  (iii).  Weobserve  that  the  equations 


(4.1.2) 


,Va  ♦  Xb  =  0 
lX'a  =  X'd 


are  solvable  for  any  d,  in  which  case 


(4  1.3) 


a  =  C2  X'  d 

'b  =  -C.  X’  d 
4 


is  a  solution. 

Substituting  (4.1.3)  in  (4.1.2)  and  omitting  d,  we  have 
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(4.1.4) 


V  C-,  X'  =  X  CA  X' 
2  4 

X’  C2  X’  =  X’, 


In  view  of  (i)  we  can  replace  C^,  C2  *  •  C^,  Cl  in  (4.1.4)  to 
(4.1.5) 


V  CL  X'  =  X  Cl  X1 
a  4 


X'  C'5  X'  =  X’  . 

Multiplying  both  sides  of  the  first  equation  in  (4.1.5)  by  X 


(4.1.6) 


X  C.  V  CL  X'  =  X  C,  X  C'  X'  =  X  Cl  X* . 

3  a  3  4  4 


So  that  X  X’  is  symmetric. 

Then  (4. 1.4-6)  prove  the  results  (ii)  and  fi  i) . 
(iv)  We  observe  th;;t  the  equations 

r^Va  +  Xb  *  Xd, 

I  X’a  =  0, 


(4.1.7) 


are  solvable  for  any  d.  Then, 
i  =  Cj  Xd, 


(4.1.8) 


b  =  C.  Xd, 


is  a  solution.  Substituting  (4.1.8)  in  (4.1.7)  and  omitting 
(4.1.5) 


V  Cj  X  *  X  C,  X  a  X. 


X’  C1  X  =  0, 


But  X  C3  X  =  X  =>V  Cl  X  =  0. 

Also,  V  C|  X  =  0  in  view  of  (i). 

The  result  VC.  V  ♦  X  Cj  V  =  V  easily  follows. 


(V)  We  observe  that  the  equations 
Va  +  Xb  =  Vd, 


(4.1.10) 


X’a  =  0, 


obtain 


C„,  we  obtain 
a 


d,  we  have 


are  solvable  for  any  d.  One  solution  is 
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(4.1.11) 


a  =  Cj  Vd, 
h  =  c3  Vd. 


Substituting  in  (4.1.10)  and  omitting  d 
(4.1.12; 


V  Cj  V  +  X  c3  V  =  V. 


X’  Cj  V 


=  0. 


This  implies  that 

(4.1.U)  V  Cj  V  Cj  V  .  V  Cj  X  Cj  V  -  V  Cj  V  .  V  C,  V  Cj  V, 
since  V  Cj  X  =  0. 

Also,  since  V  Cj  X  =  0, 

A 

(4.1.14)  V  CJ  V  Cj  V  =  V  CJ  V, 

and  V  Cj  V  is  symmetric. 

(vi)  To  prove  (vi)  we  use  the  result 


*  Tr(V  Cj)  ♦  R(X  C3)  +  R(X*  C2) 


(4.1. 15)  *  Tr(V  Cj)  +  R(X)  +  R(X'). 
Moreover, 

(4.1.16) 


=  R(V:X)  +  R(X) . 

Equating  (4.1.15)  and  (4.1.16)  we  have 
Tr  V  Cj  =  R(V:X)  -  R(X) . 

The  results  (vii)  and  (viii)  are  proved  by  direct  verification. 


dNK 
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Remark  4.1.1 

(  i)  The  results  (ii),  (iii)  and  (iv)  of  above  theorem  are  necessary 

and  sufficient  for  relation  (4.1.1)  to  hold. 

(  ii)  C2  and  Cl  are  in  fact  minimum  V-norm  g-inverse  of  X'. 

V 

(iii)  (C|:C^)  is  a  minimum  V-norm  g-inverse  of  f<(). 

As  remarked  earlier  the  inverse  matrix  (4.1.1)  is  like  a  Pandora’s  Box 
which  gives  all  that  is  necessary  for  drawing  inference  on  the  8-parametpr<- 
We  state  the  results  in  Theorem  4.2.1  which  demonstrates  the  use  of  the  sub 
matrices  in  (4.1.1). 

4.2  Main  Results 
Theorem  4.2.1 

Let  C^,  C2>  C_,  C4  be  as  defined  in  (4.1.1).  Then  the  following  hold: 

(  i)  [Use  of  C2  or  C3j .  The  BLUE  of  an  estimable  parametric  function 
p' 8  is  p' 8  where 

(4.2.1)  8  =  C'  V  or  3  =  Cj  Y 

;  ii)  [Use  of  C^].  The  dispersion  matrix  of  8  is  o“  in  the  sense, 

(4.2.2)  Var(p’B)  =  a"  p‘  C4  p, 

(4.2.3)  Cov(p'6,  q'8)  =  o2p'C4q  =  o2q'C4p, 
where  p’8  and  q’8  are  estimable. 

2 

(iii)  [Use  of  C^].  An  unbiased  estimator  of  a  is 

(4.2.4)  n2  -  f"1  Y’  Cj  Y 
where 


f  =  R(V: X)  -  R(X) 
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Proof: 

(  i)  If  L'Y  is  an  unbiased  estimator  of  p'B  then 
X’L  =  p. 

Subject  to  this  condition 
V(L'Y)  =  a2L'VL, 

or  L’VL  has  to  be  minimized  to  obtain  the  BLUE  of  p'B. 

Let  L*  be  an  optimum  choice  and  L  be  any  other  vector  such  that 
X'L  =  X’L*. 

Then 

L'VL  =  (L-L^  +  LJ  '  V(L-L*  +  L*) 

=  ( l-l*) 1  v(l-l*)  +  l;vl*  +  2l;v(l-lj  >_  l;vl*, 

iff  L*V(L-L*)  =  0  whenever  X'(L-L*)  =  0;  i.e.,  VL4  =  -XK* 
for  a  suitable  K*. 

Then  L*  and  K*  satisfy  the  equations 


(4.2.5) 


,VL*  +  XK*  =  0 
X'L*  =  p 


We  observe  that  the  equations  (4.2.5)  admit  a  solution  and  any  ‘wo 
solutions  Lj*  and  L7*  satisfy  the  condition 

V(Lj*  -  L2J  =  0. 

Since  (4.2.5)  is  consistent,  a  solution  is  given  by 


L*  =  C2p  L*  =  C 

or 

=  K*  =  *C^p 

Then  the  BLUE  of  p'B  is 

L;y  =  p’C^Y  =  p'C3Y. 

(  ii)  We  use  the  fact  that  p  =  X’M  for  some  M.  Then 
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Var(p'C^Y)  =  c2M' (XC^V)C2X'M 

'i 

=  o^M'XC^fX'C^X'jM  Using  Theorem  4.1.1  (iii) 

=  o“M' XC^X'M  Using  Theorem  4.1.1  (  ii) 

2 

=  o  p'C4p. 

Similarly, 

Cov(p'C'Yq'C'Y)  =  a2p'C4q  =  a2q*.:4p. 

(iii)  Since  X'C-V  =  0  and  X'CjX  =  0,  using  Theorem  4.1.1  (  iv) , 

Y'CjY  =  (Y  -  X8)!  C^Y-XS). 

We  have 

H l (Y-X3)  '  (Y-X8)  ]  =  o2  Tr  C3  [E{  (Y-XB)  (Y-XB)’)] 

=  a2  Tr  C3V  =  o2[R(V:X)  -  R(X)j, 
where  the  last  equality  follows  from  (vi)  of  Theorem  4.1.1. 

Theorem  4.2.2 

Let  P'B  be  the  vector  of  BLUE's  of  a  set  of  k  estimable  parametric 

functions  P'8,  Rq  =  Y'C.Y  and  f  be  as  defined  in  Theorem  4.2.1.  If 

Y  -  N  (X8,o2V),  then: 
n 

(  i)  P'B  and  Y’C,Y  are  independently  distributed  with 

(4.2.6)  P’§  -  NV(P’8,  o2D) 
and 

(4.2.7)  Y'C,Y  -  c24 

where  D  =  P'C.P. 

4 

(ii)  Let  P'8  =  w  be  the  null  hypothesis.  The  null  hypothesis  is  consistent  iff 

(4.2.8)  DD~u  =  u 
where  u  =  P'8  -  w. 
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If  the  hypothesis  is  consistent,  then 

R2 

(4.2.9)  F  =  *  -2.  ,  h  =  R(D) 

h  t 

has  a  Central  F  distribution  on  h  and  f  degrees  of  freedom  when  the 
hypothesis  is  true,  and  a  non-central  F  distribution  when  the  hypothesis 
is  false. 


Proof  (i) 

The  result  (4.2.6)  is  easy  to  establish. 
(4.2.7)  follows  since 

c  ♦  c;  c.  ♦  c; 

Y'(— -)Y  =  (Y-XS?  (-——--)  (Y-X6) 

and  by 


VC^CjV  =  V^V 


and 


VCJVCjV  =  VCjV 

2 

which  is  an  NAS  condition  for  ax  *  dist.  (See  Rao  [3]  p.  188  and  also 
Rao  and  Mitra  [4].) 

9 

The  degrees  of  freedom  of  the  x~  is 

Tr  VCj  =  R,V:X)  -  R(X)  =  f,  using  Theorem  4.1.3,  re-jlt  (vi). 

Since  P'8  is  estimable, 

P'  -  QX  for  some  Q. 

Then  P'S  =  QXC  Y. 

C1  +  C1 

The  condition  for  independence  of  Y'( - ^ - )  Y  and  QXC^Y  is 

C  +  C' 

V(— — — »)  VC’X'Q’QXC3V  =  0 


which  is  true  since 


V(- 


-)  VC^X' 


ci  +  ci 

V(.i  -A) 


XC3V 


Using  Theorem  4.1.1  (ii)  and  (iii). 


0 


2 
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(ii)  The  hypothesis  P'S  =  w  is  consistent  for  any  vector  M. 
Var[M'(p'B-w)]  =  0  =>M’(P'6-w)  =  0, 

i.e.,  M'DM  =  0  =  M'u  =  0  or  u  ^4TD) .  for  which  a  NAS  condition 
is  DL>  u  =  u,  for  any  g-inverse  D  '  f  !). 

Since  dispersion  matrix  of  u  =  a“D  and 

DD'H  =  D, 


u'D  u 

2 


h  =  R(D), 


2  2 
Using  the  result  proved  in  (i),  is  distributed  as  Xg 

independently  of  u.  Hence  the  result  (4.2.9)  follows. 


Q.E.D. 


In  Theorem  4.2.2  the  numerator  of  the  F  statistic  for  testing  the 
linear  hypothesis  p'S  =  w  was  obtained  in  the  form  U'DU  which  involved  the 
estimation  of  deviations  in  individual  hypotheses,  computations  of  their 
dispersion  matrix  and  its  inverse. 

Theorem  4.2.5  provides  an  alternative,  method  of  computing  the  numerate 
as  in  the  theory  of  least  squares. 


Theorem  4.2.3 

Let  Cj  be  as  defined  in  (4.1.1)  and 


for  any  choice  of  the  g-inverse.  Further  let  Y  have  a  MVN  distribution. 
Then  the  hypothesis  P’S  =  w  is  consistent  iff 
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in  which  case 


Ti  =  Cw)  Ei(  w  ]-Y’ciY  and 


T2  =  Y'CjY 


are  independently  distributed  as  o"x~  on 


d  =  R  (o  p.l  '  K(V:X)  and 


f  =  R ( V : X)  -  R(X) 


degrees  of  freedom  respectively. 
Hence, 


has  the  F  distribution  on  d  and  f  degrees  of  freedom. 


5 •  unified  LEAST  SQUARES  METHOD  (ULS) 

5 . 1  Statement  of  the  Problem 
Suppose  we  lave  a  GGM  model 

(5.1.1)  (Y,X3,o2V). 

(a)  When  V  =  I  and  X  is  of  full  rank  in  (5.1.1)  Gauss  [2]  propounded  t!>* 
famous  theory  of  least  squares  which  postulates  that  the  best  estimate 
8  of  6  is  obtained  by  minimizing  the  sum  of  squares 

(5. 1.2)  (Y-X8) '  (Y-X8) 

Gauss  showed  m  fact  that  8  is  the  BLUE  of  8  and  that  an  unbiased 
estimator  of  2  is 

(5.1.3)  a 2  =  (Y-XB) '  (Y-X8)/n-r  with 


r  =  R(X). 


to  which  the  theory  of  least  squares  is  applicable. 

Yhus  we  are  led  to  minimize 
'5.1.6}  (Yt-Xt8)’  (Yt-XtS)  =  (Y-X8) *  V_i  (Y-XB) 

which  is  the  procedure  proposed  by  Aitken  [1]. 

(c)  If  V  is  singular,  Aitken's  procedure  fails  as  V  *  does  not  exist, 
(e.g.  if  V  is  symmetric  and  n.n.d.  with  R(V)  =  r  <  n) . 

In  such  a  case  V  has  the  following  spectral  decomposition 


V  =  a.p.p;  +...+  X  P  ?’ 
Ill  r  r  r 


Let  P  j,...,P  denote  eigenvectors  corresponding  to  0  eigenvectors 

-in 

Suppose  F.  =  i  =  1 ,2, _ ,r. 

Then  Var(FIY)  r  '/P.'VF.  =  g2a'!p!VP.  =•  o2- 

i  •  i  ill 

Cov(F.XF.Y)  =  o2P.'VP.  =  0. 

i  j  i  i 

Also  let  B.  =  P j  =  1,2,... ,n-r,  then 

Var(B'.Y)  =  B'VB.  =  0. 

J  J  J 

Letting 


F  =  ^ Fi »  F2’  —  ’ Fr^  ’ 

B  =  (3,  ,  »Bn  r) , 

2  7 

the  given  model  (Y,XB,o  V)  is  reduced  to  fF'Y,  F’X6,o  I_)  with 

1 


constraints  3’X3  =  B’Y  =  C. 
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Also,  as  before  we  can  calculate 

(1)  R?  =  min  (Y-XS)'  V~(Y-X8), 

B'XS  =  C 

(2)  =  min  (Y-X8)’  V‘(f-X6). 

B'X8  =  C 
P'S  =  W 

(3)  The  F  statistic  (4.2.9)  for  testing 
Hq:  P ' S  =  W  is 

B'-R1 

K1  R0  .  •‘0 
h  v  f  • 


Motivated  by  the  knowledge  that  Aitken's  procedure  fails  when  V  is  singular, 
we  raise  the  following  ques;ion.  Does  there  exist  a  matrix  M  regari!:-  - 
whether  ’  is  singular  or  not  such  that  the  following  conditions  hc-i'i'' 

(?j  The  BLUE  of  any  estimable  (pe/tX’))  parametric  function  p'B  ic  r',‘1 
where  S  is  a  stationery'  point  of  the  function 

(5.1.10)  (Y-X3)'  M(Y-X6) 

i.e.  where  the  derivative  of  (5.1.10)  with  respect  to  8  vanishes  is  rc- 

(b)  An  unbiased  estimator  of  a  is  obtained  as 

(5.1.11)  o2  =  (Y-XS) 1  M('.-XB)  ;  f 
where 

f  =  R(V: X)  -  R(X) . 

(c)  R^  =  stationary  value  of 

(5.1.12)  (.Y-XS)*  M(Y-XS) 

under  the  restriction  P’8  =  K, 

Rq  =  stationary  value  of 

(5.1.13)  (f-XS)’  M(Y-X8) 


27 


Remark  5.1.2 

It  can  be  assumed,  M  may  be  chosen  to  be  symmetric.  Theorem  5.2.1 
provide  complete  answers  to  the  questions  (a)  and  (b).  We  show  that  ’> 
singular  or  not,  the  choice  should  be 

(5.1.14)  M  =  (V  +  XUX’)‘ 

for  any  symmetric  g-inverse  where  U  is  any  symmetric  matrix  such  that 

^r(v--x)  =^(V  +  XUX’). 

In  particular  we  can  always  choose 

(5.1.15)  M  =  (V  +  k2XX’)' 

for  any  choice  of  g-inverse  where  k  is  an  arbitrary  non-2ero  constant.  ( 
hold  for  any  choice  of  M  for  all  testable  hypotheses. 

S . 2  Some  Prill  mi nary  Lemmas 

Lemma  5.2.1  Let  T  be  a  matrix  such  that  R(X'TX)  =  R(X) .  Then 

(5.2.1)  X(X'TX) ~  (X'TX)  =  X 

Proof: 

If  R(  1'TX)  =  R(X),  then  for  any  vector 

X’TXX  =  0  if  and  only  if  XX  =  0. 

This  result  together  with  the  identity 

(5.2.2)  0  =  X'TX(X'TX) "  X'TX  -  X'TX  =  X’TXf(X’TX)-  X'TX  -  I], 
yields  (5.2.1). 

Lemma  5.2.2  Let  (J  be  symmetric  and  V  be  n.n.d.  matrices  such  that 
(5.2.5)  _^(V:X)  =  ^(V  ♦  XUX’) 

Then 

R(X’(Y  *  XUX')”  X]  =  R(X') 

“roof: 

The  result  is  easy  to  establish  using  Lemma  5.2.1. 


9 


cl  cannot 
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Lemma  5  . 3  Let  X6  be  the  BLUE  of  XS.  Then  the  unbiased  estimator  -r  js 

(5.2.4)  f" 1 (Y-XB) '  V~  (Y-XS)  =  f' 1 (Y-XB) ’ (V  +  XUX*)’  (Y-XB) 
where 

f  =  R(V:X)  -  R(X)  and  U  is  defined  in  Lemma  5.2.2. 

Proof : 

The  left-hand  side  expression  in  (5.2.4)  is  well -known. and  the  equivalent' 
with  the  right-hand  side  follows  easily  observing  that  Y-XS  €_^f(V) . 

Theorem  5.2.1 

Let  (Y.XB.o^V)  be  a  GGM  model  and  M  be  a  symmetric  matrix  such  that 
X 1  MV)  C^(X’MX) 

in  which  case 

(Y-XB)'  M(Y-XB) 

as  a  function  of  2  has  stationary  values.  Further  let  B  be  a  star*  -nt. 

If  p'B  is  the  BLUE  of  p’B  for  every  p§4fX’)-  Then  it  is  necessary 

(5.2.5)  R(X'MX)  =  R(X) 
and  M  is  of  the  form 

(5.2.6)  (G  ♦  XUX’)"  <-  K 

for  any  symmetric  choice  of  g-inverse  where  U  and  X  are  any  symraetrit.  mv .  ■  es 
such  that 

(5-2-7)  ^(V:X)  =^(V  ♦  XUX’) 

(5.2.8)  VKX  =  0  X’KX  =  0 

Conversely:  If  M  is  of  the  form  (5.2.6)  with  (5.2.7)  and  (5.2.8)  *riK,  -in 
R(X’MX)  =  R(X)  and  p'S  is  the  BLUE  of  p'B 
for  every  pi^X') . 

Proof : 

Equating  the  derivative  of  (Y-Xg)’M  (Y-Xg)  to  zero,  we  obtain 

(5.2.9)  X'MXB  =  X'MY 
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which  is  consistent  since^/'(X'MX)  and  Y€^ftV:Y)  with  probabili*.  1 

In  this  case 

(5.2.10)  f 3  =  (X’MX)‘  X'MY 
is  a  stationar)'  point. 

Let  p  =  X’L.  Then  p’B  is  the  BLUE  of  p'S  and  it  follows  by  derinitiii.  •. 

(5.2.11)  L' X(X'MX) "  X 1  MX  =  L’X. 

Since  L  is  arbitrary  in  (5.2.11),  we  have 

X(X'MX) "  ( X * MX)  =  X  =>  R(X'MX)  =  R(X), 
which  proves  (5.2.5)  . 

If  p'B  is  the  BLUE  of  p'$  for  every  pi^X')  then  applying  the  lemma  or  r 
of  Rao  (5],  we  have 

(5.2.1?)  L'X(X'MX)'  X'MVZ  =  0  for  any  L, 

where  Z  is  a  matrix  of  maximum  rank  such  that 
X'Z  =  0. 

Then  (5.2.12)®  X(X’MX)'  X'MVZ  =  0°  X'MVZ  =  0 

(5.2. 13)  ®  VMX  =  XQ 
for  some  Q. 

Now  there  exists  a  symmetric  matrix  U  such  that 

(5.2.14)  X'M(V  +  XUX ' )  MX  =  X'MX. 

Let  W  -  X'MX.  Then  it  can  be  verified  easily  that  one  choice  of  U  ir 
W"  (-X'MVMX  +  W)W‘, 

where  W  is  a  symmetric  g-inverse  of  W. 

Multiplying  both  sides  of  (5.2.14)  by  X(X'MX)~  and  using  (5.2.13)  and  Lc  r*: 
we  obtain 


(5.2.15) 


(V  +  XUX') MX  =  X. 


If  a'  (V  +  XUX')  =  0  then  from  (5.2.15)  a’X  =  0  and  hence  a'V  =  0  and 
vice-versa,  which  proves  (5. 2. 7) .Choosing  a  symmetric  g-inverse  s nd  a 
symmetric  K,  let 


(5.2.16) 


M  *  (V  -  Xl'X')"  +  K. 


Substituting  (5.2.16)  in  (5.2.15),  wo  obtain  (5.2.8). 

The  converse  is  easy  to  prove  using  Lemma  5.2.2. 

Theorem  5.2.2 

Let  8  be  a  stationary  point  of  (Y-XB) 'M(Y-XB)  where  M  is  a  symmetric 
matrix  such  that 

^f(X'MX)  34KX'MV). 

If  p'S  is  the  BLUli  of  p'B  for  every  p§4(X*)  and  for  all  Y£$V:X), 


(5.2.17) 


a2  =  f_1(Y-XB)'M(Y-X8) 


is  an  unbiased  estimator  for  a  ,  then  it  is  necessary  and  sufficient  th.- 
M  is  a  symmetric  g-inverse  of  V  +  XUX'  where  U  is  any  symmetric  matrix  * 
that^(V:X)  =^(V  ♦  XUX'). 

Proof: 

We  have  already  seen  that  M  is  of  the  form  (5.2.6)  and  K  satisfies  (5.. 
If  (5.2.17)  is  the  same  as  (5.2.4),  then 
(Y-XS) ' K(Y-X8)  =  0  =  Y’KY. 

ing  (5.2.8)  for  al!  Y^itV  +  XUX'),  which  implies  that  VKV  =  0  in  addit 
to  (5.2.8).  Then, 

(V  +  XUX')[(V  +  XUX')"  +  K] (V  +  XUX’)  =  (V  +  XUX') 
which  shows  that  M  is  a  g-inverse  of  (V  +  XUX'). 

Remark  5.2.1 

(  i)  In  Theorem *5. 2. 1  we  showed  that  M  is  a  symmetric  g-inverse  of 


(V  +  XUX').  It  may  be  seen  that  the  expression 
(Y-XS) ' (V  ♦  XUX')"  (Y-XB) 
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is  independent  of  the  choice  of  a  g-inverse  and  in  practice  one  can  use 
any  g-inverse. 

(ii)  If  V  -t  XUX'  is  a  n.n.d.  matrix  then 

(Y-XS)'(V  +  XUX')"  (Y-XB) 

is  independent  of  the  choice  of  the  g-inverse,  is  non-negative  and  obtains 
a  minimum  at  6  where  the  derivative  vanishes. 

We  can  always  choose  U  in  such  a  way  that  V  +  XUX*  is  n.n.d.  and  satisfies 
(5.2.7).  For  example  U can  be  any  p.d.  matrix. 

(iii)  It  may  be  seen  that  (V  +  XUX')"  need  not  be  a  g-inverse  of  V. 

If  there  exist  a  matrix  U  such  that  V  +  XUX*  satisfies  (5.2.7)  and 

'(V  -  XUX')"}  c  (V‘> 

then  it  can  be  shown  that  a  NAS  condition  is: 

jtiy)  rue(x.jx*)  =  (ok 

Such  a  choice  of  U  can  be  made  if  necessary. 

Theorem  5.2.3 

Let  M  be  chosen  as  in  Theorem  5.2.2  and  P'8  be  a  set  of  k  estimable  fue<'* 

wK(P)  c  jt(X'). 

Then  P'8  are  the  BLUE's  of  P'B  and  the  dispersion  matrix  of  P'8  is 

(5.2.18)  D(P'3)  =  o2p'[(X'(V  +  XUX')'  X)"  -  U]P 

Proof: 

Let  W  =  (V  +  XUX'). 

Then 

P'8  =  P'(X'W  X)'  X’W'Y 

and 

(5.2.19)  D(P' 8)  =  o2P'(X'W~  X)-  X'W'  V[P'(X’W"  X)*  X'tf"; 

Write  V  =  V  +  XUX'  -  XUX’ 


'•ns  i .  e 


=  W  -  XUX'  in  (5.2.19). 


Then,  repeatedly  using  the  relation  (a)  of  Section  2.2  we  t  (5.2.18). 
Finally,  to  test  the  hypothesis 

P'8  =  W. 

We  proceed  as  follows: 

Let  u  =  p'g  -  W  and 

D(u)  =  o2D,  R(D)  =  h. 

The  hypothesis  is  consistent  if 
(5.2.20)  D[)~  u  =  u. 

If  (5.2.20)  ho’ds,  then  the  null  distribution  of  the  statistic 


(5.2.21) 


F  = 


u'D  u 


is  the  F  distribution  on  h  and  f  degrees  of  freedom  when  Y  -  MVN(XB,o2V) . 
The  results  (5.2.20)  and  (5.2.21)  are  proved  in  section  4. 

In  Theorems  5.2.1  and  5.2.2,  it  is  shown  that  there  exists  a  matrix  M, 
whether  V  is  nonsinguiar  or  not,  such  that  a  stationary  value  B  of 


(Y-X8) 'M(Y-XB) 

provides  the  BLUE  of  an  estimable  function  p'B  as  p'B,  and  an  unbiased 
? 

estimator  of  o~  is 

o2  =  f”1 (Y-XB) 'M(Y-XB) . 


So  far  we  have  an  analogue  of  the  ieast  squares  theory  in  the  general  care. 

The  first  departure  from  the  least  squares  results  is  the  expression 

(5.2.18)  for  the  dispersion  matrix  of  P'8,  which  contains  the  extra  term 
2 

o^P'UP.  It  can  be  shown  that  there  exists  no  choice  of  M  unless^(x)  c.J£{\') 
such  that 

(5.2.22)  0{P*  8'  =  o2P'(X'MX)_P 

for  all  P  such  that  P'S  is  estimable. 


Since  (5.2.22)  does  not  hold,  there  exists  no  choice  of  M,  unless 
_^(x)  c  _^(V) ,  which  enables  the  computation  of  the  numerator  of  the  F- 
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statistic,  (5.2.21),  in  the  form 

u'Du  =  min  (Y-XB) 'M(Y-XB)  -  min(Y-XS) ’M(Y-XB) 

P’S=w 

for  all  testable  hypotheses  of  the  form  P'B  =  w. 

However  Rao  [6]  and  Mitra  [10]  have  shown  that  a  suitable  choice  of 

M  can  be  made  provided  the  null  hypothesis  is  written  in  a  modified  but 

an  equivalent  form.  The  computation  of  such  an  M  is  somewhat  complicated 

and  it  is  much  simpler  to  compute  the  F-statistic  as  in  Theorem  5.2.3,  using 

the  simple  choice  of  M  as  in  Theorems  5.2.1  and  5.2.2  for  estimating  P'B-w 
2 

and  a  .  Note  that  M  can  always  be  chosen  as  (V+XX’)  ,  which  satisfies  the 
conditions  of  the  Theorem  5.2.2  (see  Rao  and  Mitra  [12]). 

6.  BLUE'S  AS  PROJECTIONS 
6. 1  Projection  Operators 

It  is  well  known  that  when  V  is  r.onsingular  the  BLUE  of  XB  is  obtained 
by  the  orthogonal  projection  of  Y  on_4$x),  using  the  norm  j]xi|=  (x'V  x)  “, 
which  is  the  same  as  the  projection  of  Y  on_^(x)  along^(VZ),  where  Z  =  X  . 
[Note  that^Tx)  and.^g'fVZ)  are  disjoint  subspaces  whether  V  is  nonsingular  or 
not].  We  prove  the  corresponding  results  when  V  is  singular.  Naturally, 
the  results  have  to  be  stated  in  a  slightly  different  manner  since  V  *  does 
not  exist  (hence  the  norm  ||x|j  cannot  be  defined  as  in  the  nonsingular  case), 
and^f(x)  and_£(VZ) ,  although  disjoint,  may  not  span  the  entire  space  F.n(hence 
the  projection  or.^f(x)  along^(VZ)  is  not  properly  defined). 

Definition  6. I . i .  Let  Z  be  an  n.n.d.  (non-negative  definite)  matrix 
of  order  n  and  define  E-norra  as 

j  jx{ \z  =  (x’Ex)1/2. 


(6.1.1) 
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Further  let  A  be  an  nxm  matrix.  We  call  P  ^  a  projector  into>f(A)  under 
the  E-norm  if 


f  MP 


AE>  C  ^A) 


(6.1.2) 


l  l|y-pAi!!E  i  I  |y  -  AX II E  for  all  y€En,  ,\€Em. 

The  following  lemma  is  easily  established  (see  Mitra  and  Rao,  [11]). 
Lemma  6.1.1.  If  P  is  as  defined  in  (6.1.2),  it  is  necessary  and 
sufficient  that 


(6.1,3) 


MPkl)  cJX A) 


(6.1.4) 

(6.1.5) 


V*  PAE  *  EPAE  =  <PA£>'E 


EP  A  =  EA 
AE 


Definition  6.1.2.  Let  U  and  W  be  two  matrices  such  that  ^£(U)  and 
^ftW)  are  disjoint,  which  together  may  not  span  the  entire  space.  Any 
vector  a(i<£fU:W)  has  the  unique  decomposition 

a  =  ctj  *  a2-  a3  €^f(l!) ,  a.>  6_>tfW) . 

Then  is  said  to  be  a  projector  onto>£(U)  along^f(W)  iff 

(6.1.6)  P^j>A>  o  =  cij  for  all  a  6^(U:W). 

The  following  lerrnia  is  easily  established. 

Lemma  6.1.2.  If  is  a  projector  as  given  in  Definition  2,  then 

it  is  necessary  and  sufficient  that 

(6.!. 7)  PU|K  u  =  u,  r0|w  w  .  0 

and  one  choice  of  Pyjw  is 

(6.1.8)  Py.  =  U(GU)‘G. 

where  G’  =  Wx  and  (GU)  is  any  g-inverse  of  GU. 
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6.2  Applications 

The  following  theorem  provides  expressions  for  the  BLUE'S  in  terms 

of  projection  operators  as  described  in  definitions  1  and  2. 

2 

Theorem  6.2.1.  Consider  the  G.G.M.  model  (Y,  XB,  o  V) .  Then  the 
following  hold: 

(  i )  Let  L'Y  be  an  unbiased  estimator  of  p'B  with  the  property  L'X  =  p1, 

and  define  L*  =  (I-P^y)L,  where  Z  =  XX.  Then  Li  Y  is  the  BLUE  of  p'B. 

(  ii)  Let  S  =  V+XX1,  S~  be  any  n.n.d.  g-inverse  of  S,  and  Z  -  Xx,  Then 

(6.2.1)  (P^v  +  Pxs-)  a  =  a  for  any  a  €_i*V:X) 

i.e,,  the  sum  of  the  projection  operators  on  the  left  hand  side  of 
(6.2.1)  is  an  ideitity  in  the  space  _^(V:X)  =^(VZ:X). 

(iii)  The  BLUE  o t  XS  is 

(6.2.2)  U-P|V)Y  -  CPjjs-n  -  (PX|V:)V 

where  the  projection  operators  are  as  described  in  definitions  1  and  2. 
Proof  of  (i) .  Since c^Z) ,  P£v  X  =  0  and  hence 
E(L’P’V  Y)  =  L'  •  P^v  X  =  0, 

giving 

E(L;Y)  =  E(I.'Y)  -  E(L'P^Y)  =  E(L'Y) 
so  that  LiY  is  unbiased  for  p'B.  Further 
L*  VZ  =  L'(I-PZV)'  VZ  =  0 

using  the  conditions  (7.4)  and  (7.5),  which  shows  that  LiY  has  minimum 
variance. 

Proof  of  ( i  i) .  Since  ^*C(V:X;  =^(VZ:X)  we  need  only  verify  that 

(pzv  v  pxs-}  (VZ:X)  =  (VZ:X) 

which  follows  from  the  definitions  of  the  projection  operators. 
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Proof  of  (iii).  From  (i)  it  follows  that  the  BLUE  of  X8  is 

0-pzv>Y 

and  from  (ii)  we  have 

0-P£v)Y  -  (PXS-)Y 

To  prove  the  last  part  of  the  equality  in  (6.2.2),  consider  the  unique 
decomposition 

(6.2.3)  Y  =  XY1  +  VZY2 

on  the  disjoint  subspaces_^(x)  and^(VZ).  Note  that  XY^  =  (px|y^)Y  where 
Pxjvz  is  the  projector  onto^(x)  along  ^(VZ) .  Now 

Xp  *  E(Y)  =  XEfYp  +  VZE(Y2), 

=»  xfB-EfYpj  =  VZE(Y2)  =  0 

since^x)  and^(VZ)  are  disjoint.  Then  EfXYjJ  -  E(Y),  so  that  XYj  is 
unbiased  for  xS. 

Further  from  (6.2.3) 

Cov(Y.Z'Y)  =  X  Cov(Yj ,Z'Y)  +  VZ  Cov(Y2,2’Y) 

(6.2.4)  a  VZ  =  XD1  +  VZD2  for  some  Dj  and  D2 

=>VZ(I-D2)  *  XDj  =  0  =  Cov  (XYj.Z'Y) 

which  shows  that  XY^  is  the  BLUE  of  E(XY^)  =  xB. 

Theorem  6.2.1  is  thus  completely  proved. 

Note  that,  following  (6.1.8),  we  can  represent 

(6.2.5)  Pxjy2  =  X(GX)"G 

where  C  =  (VZ)A.  When  V=I,  we  have  G  =  X'  giving  the  BLUE  of  xb  as 


(6.2.6) 


(Pxjyz)Y  =  X(X'X)  X'Y- 
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When  v  is  nonsinguiar,  we  have  G  =  X'V  *  giving  the  BLUE  of  x6  as 
(6.2.7)  PX|V,  Y  =  X(X’V'iX)'X,V'lY. 

Thus  (6.2.5)  provides  the  well  known  formulae  (6.2.6)  and  (6.2.7)  in 
the  particular  cases  considered. 

In  these  lectures  we  have  considered  the  problem  of  estimating 
p'B  by  L'Y  such  that  L'VL  is  a  minimum  subject  to  X'L  -  p,  which  provides 
a  complete  solution  to  the  BLUE.  However,  this  approach  does  not  provide 
all  possible  re,  •  c-sentations  of  the  BLUE.  For  this,  one  has  to  minimize 
L'VL  subject  to  the  condition  X'L-p  €^(X'N)  where  N  is  as  defined  in 
Section  3.1.  The  latter  problem  called  BLUE(W),  BLUE  in  wider  sense,  which 
is  of  soma  theoretical  interest  is  solved  in  Rao  (9j . 

Note.  The  references  given  at  the  end  of  the  notes  constitute  the  material 
on  which  the  lectures  were  based.  For  reference..  to  related  work  by  ether 
authors  the  reader  is  referred  to  bibliographies  in  Rao  [5]  and  [6].  It  may 
be  noted  that  Goldman  and  Zclen  [13]  were  '.he  first  to  consider  the  case  of 
nonsingular  V  in  a  systematic  •jay  using  generalized  inverses. 
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